Spatial input device

ABSTRACT

A spatial input device includes a light beam scanner that emits light beams toward a spatially projected image while two-dimensionally scanning the light beams, a photodetector that detects the light beams that have been reflected by an input object within a detection range, with the detection range extending inward of the spatial input device relative to the image, and a controller that counts a scan line number indicative of a number of the light beams that have been detected by the photodetector, and detects a depth position of the input object based on the scan line number. The controller further detects the depth position of the input object based on a depth conversion table, with the depth conversion table specifying depth layer levels corresponding to scan line numbers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation application of U.S. patent application Ser. No.14/224,457, which claims priority to Japanese Patent Application No.2013-080206 filed on Apr. 8, 2013. The entire disclosures of U.S. patentapplication Ser. No. 14/224,457 and Japanese Patent Application No.2013-080206 are hereby incorporated herein by reference.

BACKGROUND

1. Field of the Invention

The present invention generally relates to a spatial input device.

2. Background Information

Conventionally, spatial input devices with a virtual user interface arewell-known in the art (see Japanese Unexamined Patent ApplicationPublication No. 2010-55507 (Patent Literature 1), for example).

For example, the above-mentioned Patent Literature 1 discloses aninput/output device. With this input/output device, when an image isdisplayed by display elements disposed in unit regions corresponding tomicrolenses of a microlens array, the displayed image is focused in aspatial plane by the microlens array. With this input/output device,after light from an object used as an indicator, such as a finger, isconverged by the microlens array, it is received by imaging elementsdisposed in the same plane as the display elements. Then, imaging dataabout the object is acquired. The position of the object is sensed basedon this imaging data.

SUMMARY

The three-dimensional position of the object can be detected with theinput/output device of the above-mentioned Patent Literature 1. However,this device requires the microlens array or the imaging elements. Thus,the structure of the device becomes complicated.

One aspect is to provide a spatial input device with which a depthposition of an input object can be detected with a simple structure.

In view of the state of the known technology, a spatial input device isprovided that includes a light beam scanner that emits light beamstoward a spatially projected image while two-dimensionally scanning thelight beams, a photodetector that detects the light beams that have beenreflected by an input object within a detection range, with thedetection range extending inward of the spatial input device relative tothe image, and a controller that counts a scan line number indicative ofa number of the light beams that have been detected by thephotodetector, and detects a depth position of the input object based onthe scan line number. The controller further detects the depth positionof the input object based on a depth conversion table, with the depthconversion table specifying depth layer levels corresponding to scanline numbers.

Also other objects, features, aspects and advantages of the presentdisclosure will become apparent to those skilled in the art from thefollowing detailed description, which, taken in conjunction with theannexed drawings, discloses one embodiment of the spatial input device.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the attached drawings which form a part of thisoriginal disclosure:

FIG. 1A is a schematic perspective view of an image display device inaccordance with one embodiment;

FIG. 1B is a side elevational view of the image display device;

FIG. 2 is a block diagram of an infrared laser unit of the image displaydevice;

FIG. 3 is an exploded perspective view of a photodetector of the imagedisplay device, illustrating the configuration of the interior of ahousing of the photodetector;

FIG. 4 is a schematic side elevational view of the image display device,illustrating an example of setting a detection range of thephotodetector;

FIG. 5 is a timing chart illustrating an example of a verticalsynchronization signal, a horizontal synchronization signal, and a lightreception level of the photodetector;

FIG. 6 is a schematic side elevational view of the image display device,illustrating an example of setting a depth to be nonparallel to a touchsurface;

FIG. 7 is a schematic side elevational view of the image display device,illustrating an example of setting the depth to be parallel to the touchsurface;

FIG. 8 is a flowchart illustrating a touch coordinate and depthdetection processing;

FIG. 9 is a diagram illustrating an example of a depth conversion table;

FIG. 10 is a schematic diagram illustrating a dragging of an icon;

FIG. 11 is a flowchart illustrating a processing to change to drag mode;

FIG. 12 is a flowchart illustrating a processing in the drag mode;

FIG. 13 is a schematic side elevational view of an image display devicein accordance with a modified embodiment, with image display devicehaving two photodetectors;

FIG. 14 is a schematic diagram illustrating a pull-out operation; and

FIG. 15 is a flowchart illustrating a pull-out operation determinationprocessing.

DETAILED DESCRIPTION OF EMBODIMENTS

A selected embodiment will now be explained with reference to thedrawings. It will be apparent to those skilled in the art from thisdisclosure that the following descriptions of the embodiment areprovided for illustration only and not for the purpose of limiting theinvention as defined by the appended claims and their equivalents.

Referring initially to FIGS. 1A and 1B, an image display device 10(e.g., a spatial input device) is illustrated in accordance with oneembodiment. FIG. 1A is a perspective view of the image display device10. FIG. 1B is a side elevational view of the image display device 10.

The image display device 10 as shown in FIGS. 1A and 1B includes adihedral corner reflector array substrate 1 (e.g., a reflection elementaggregate substrate), a liquid crystal display component 2, an infraredlaser unit 3, a photodetector 4, and a controller 5 (see FIG. 2).

The dihedral corner reflector array substrate 1 forms in space anoptical image S of the image displayed by the liquid crystal displaycomponent 2, allowing the user to view the resulting image.

The dihedral corner reflector array substrate 1 is configured such thata plurality of through-holes, which are square in shape and pass throughthe main face of the substrate 1 in the vertical direction, are arrangedin a zigzag pattern in plan view on the substrate 1. Mirror surfaces areformed as a dihedral corner reflector on two perpendicular faces of theflat inner wall surfaces of each of the through-holes. The dihedralcorner reflector array substrate 1 can be a conventional dihedral cornerreflector array substrate. Thus, detailed description of theconfiguration will be omitted for the sake of brevity.

Light rays emitted from a point light source disposed inside a space onone main face F2 side of the substrate 1 (see FIG. 1B) are reflectedtwice by the dihedral corner reflectors, and curve while passing throughthe substrate 1. Light rays emitted from the point light source andfacing different dihedral corner reflectors are reflected twice by thosedihedral corner reflectors, and then formed into a one-point opticalimage in a space on the main face F1 side. The main face F1 side is onthe opposite side of the substrate 1 relative to the main face F2 sideof the substrate 1 on which the point light source is located.

The liquid crystal display component 2 (e.g., the liquid crystal displaypanel) surface-emits image light. However, the liquid crystal displaycomponent 2 can be interpreted as a set of the point light sources.Therefore, light rays of the image light that surface-emitted by theliquid crystal display component 2 are reflected by the dihedral cornerreflectors and form the optical image S at a position symmetrical to theliquid crystal display component 2 relative to the substrate 1.Specifically, in the illustrated embodiment, the liquid crystal displaycomponent 2 is located in the space on the main face F2 side of thesubstrate 1 (i.e., the space under the main face F2 in FIG. 1B). Thus,the light rays form the optical image S at the position symmetrical tothe liquid crystal display component 2 relative to the substrate 1 inthe space on the other main face F1 side (i.e., the space above the mainface F1 in FIG. 1B). Consequently, the user or observer has theimpression that the image is being displayed in the air.

In order for the image display device 10 to function as a virtual userinterface, the infrared laser unit 3 is provided for directing aninfrared laser light at the optical image S formed by the dihedralcorner reflector array substrate 1 as discussed above.

FIG. 2 illustrates a block diagram of the infrared laser unit 3. Asshown in FIG. 2, the infrared laser unit 3 includes an infrared LD(Laser Diode) 3A, a collimator lens 3B, a beam splitter 3C, aphotodetector 3D, a horizontal MEMS (Micro Electro Mechanical Systems)mirror 3E, a vertical MEMS mirror 3F, a laser control circuit 3G, amirror servo 3H, and an actuator 31.

The infrared LD 3A emits a red laser light at a power level controlledby the laser control circuit 3G. The infrared laser light thus emittedis made into a parallel beam by the collimator lens 3B, and is incidenton the beam splitter 3C. A part of the light that is incident on thebeam splitter 3C is reflected and received by the photodetector 3D. Onthe other hand, the other part of the light is transmitted and headstoward the horizontal MEMS mirror 3E. The laser control circuit 3Gcontrols the output power of the infrared LD 3A based on the detectionsignal produced by the photodetector 3D.

The laser light incident on and reflected by the horizontal MEMS mirror3E is incident on and reflected by the vertical MEMS mirror 3F. Thehorizontal MEMS mirror 3E deflect the laser light so that it scans inthe horizontal direction. The vertical MEMS mirror 3F deflects the laserlight so that it scans in the vertical direction. Then, the laser lightis emitted to the outside of the housing of the infrared laser unit 3.

The infrared laser unit 3 is disposed in a space on the main face F2side (see FIG. 1B) of the dihedral corner reflector array substrate 1 onwhich the liquid crystal display component 2 is located. Thus, theinfrared laser light emitted from the infrared laser unit 3 goes throughthe through-holes in the dihedral corner reflector array substrate 1 andis emitted to the optical image S.

Deflection by the horizontal MEMS mirror 3E and the vertical MEMS mirror3F causes the laser light emitted from the infrared laser unit 3 to betwo-dimensionally scanned with respect to the optical image S.

The mirror servo 3H deflects the horizontal MEMS mirror 3E by drivingthe actuator 31 according to a horizontal synchronization signal fromthe controller 5. The mirror servo 3H also deflects the vertical MEMSmirror 3F by driving the actuator 31 according to a verticalsynchronization signal from the controller 5.

The photodetector 4 is used to detect the laser light emitted from theinfrared laser unit 3 and reflected by an input object O, such as afinger and the like. The photodetector 4 is located in the space on themain face F2 side (see FIG. 1B) of the dihedral corner reflector arraysubstrate 1 on which the liquid crystal display component 2 is located.

FIG. 3 is an exploded perspective view of the configuration of theinterior of the housing of the photodetector 4. As shown in FIG. 3, thephotodetector 4 includes a light receiving element 4A, a converging lens4B and a flat masking member 4C. The light receiving element 4A, theconverging lens 4B and the flat masking member 4C are disposed insidethe housing of the photodetector 4. The light receiving element 4Adetects irradiation by reflected laser light. The converging lens 4Bconverges the reflected laser light incident from the window of thehousing, and guides it to the light receiving element 4A. The flatmasking member 4C is disposed between the light receiving element 4A andthe converging lens 4B, and is tall enough to cover part of the lightreceiving element 4A.

As shown in FIG. 2, the light receiving element 4A is connected to thecontroller 5, and sends the controller 5 the detection signal producedby the light receiving element 4A.

The masking member 4C has a width in the same direction as the widthdirection of the light receiving element 4A. The masking member 4C has acurved shape such that its two ends are closer to the converging lens 4Bside than the middle part. The masking member 4C restricts irradiationof the light receiving element 4A by blocking the reflected laser lightaccording to the incidence angle on the light receiving element 4A.

The diameter of the spot of the reflected laser light converged by theconverging lens 4B and directed to the light receiving element 4A islarger in the ends in the width direction of the light receiving element4A than the middle part. Therefore, without the masking member 4C, thereis the risk that the part of the reflected laser light that is supposedto be blocked by the masking member 4C is not all be blocked because ofthe increased spot diameter. As a result, the part of the reflectedlaser light is instead be received by the light receiving element 4A andmistakenly detected. In view of this, the masking member 4C has a curvedshape so that the reflected laser light at the ends, which has thelarger spot diameter, can be blocked while the spot diameter is stillsmall.

The detection range R1 of the photodetector 4 can be adjusted byadjusting the dimensions of the masking member 4C. FIG. 4 illustrates anexample of setting the detection range R1 of the photodetector 4. Thebroken lines in FIG. 4 indicate the boundaries (ends) of the detectionrange R1 of the photodetector 4. As shown in FIG. 4, one end of thedetection range R1 of the photodetector 4 is set to be parallel to theoptical image S near the optical image S. This end is a touch surface Tfor detecting the position where the input object O (e.g., the finger inFIG. 4) has touched the optical image S. The touch surface T cancoincide with the optical image S.

The detection range R1 of the photodetector 4 extends towards the farside of the touch surface T as seen from the advance direction (e.g.,the approach direction) in which the input object O moves into theoptical image S.

FIG. 5 illustrates an example of a vertical synchronization signal and ahorizontal synchronization signal. In the example in FIG. 5, thevertical synchronization signal is s stepped signal, while thehorizontal synchronization signal is a triangular wave signal thatincreases or decreases at each step of the vertical synchronizationsignal. These synchronization signals deflect the horizontal MEMS mirror3E and the vertical MEMS mirror 3F, so that the laser light emitted fromthe infrared laser unit 3 goes back and forth in the horizontal scanningdirection (see FIG. 4) while scanning over the optical image S in thevertical scanning direction (see FIG. 4).

When the vertical scanning direction is from top to bottom, asillustrated in FIG. 4, first the laser light is reflected by thefingertip and detected by the photodetector 4. As the scanning proceeds,the laser light is successively reflected by the bottom of the fingerand detected by the photodetector 4, until finally the laser light isreflected by the bottom of the finger and detected by the photodetector4 near the touch surface T. At this point, as shown in FIG. 5, a groupof light reception levels by the photodetector 4 appears along the timeseries of the synchronization signals. The controller 5 (see FIG. 2)detects, as a touch coordinate, the irradiation position coordinate ofthe laser light at the touch surface T based on the values of thevertical and horizontal synchronization signals (that is, the positionof the MEMS mirrors) corresponding to the light reception level finallydetected out of this group. The touch coordinate indicates the positionof the optical image S touched by the input object O. The touchcoordinate is a two-dimensional coordinate having a vertical coordinateand a horizontal coordinate.

Alternatively, when the vertical scanning direction is from bottom totop, then the touch coordinate can be detected based on the timing ofthe first detection out of the group of light reception levels.

In the illustrated embodiment, the controller 5 includes a microcomputerwith a control program that controls various parts of the image displaydevice 10. The controller 5 can also include other conventionalcomponents such as an input interface circuit, an output interfacecircuit, and storage devices such as a ROM (Read Only Memory) device anda RAM (Random Access Memory) device. The microcomputer of the controller5 is programmed to control various parts of the image display device 10.The storage devices of the controller 5 stores processing results andcontrol programs. The controller is operatively coupled to various partsof the image display device 10 in a conventional manner. The RAM of thecontroller 5 stores statuses of operational flags and various controldata. The ROM of the controller 5 stores the programs for variousoperations. The controller 5 is capable of selectively controllingvarious parts of the image display device 10 in accordance with thecontrol program. It will be apparent to those skilled in the art fromthis disclosure that the precise structure and algorithms for controller5 can be any combination of hardware and software that will carry outthe functions of the present invention. Furthermore, it will be apparentto those skilled in the art from this disclosure that the controller 5can perform the processings described below with a plurality ofmicrocomputers or processors, respectively, as needed and/or desired.

The controller 5 counts the number of scan lines (e.g., the scan linenumber), which is the number of times the light reception level isdetected, except for the last time out of the above-mentioned group,that is, the number of laser lights reflected by the input object O,such as the finger. Then, the controller 5 detects the depth positionbased on the counted number of the scan lines. Of course, alternatively,all of the detections in one group can be used. For example, a tablethat specifies depth layer levels corresponding to the number of scanlines can be stored ahead of time in a memory of the controller 5, andthe controller 5 can detect the depth layer level detected based on thistable. For example, if the number of scan lines is from 0 to 5, forinstance, then the depth layer level including the touch surface T canbe specified as 0. If the number of scan lines is from 6 to 10, then thedepth layer level can be specified as 1 that is deeper than the depthlayer level 0. If the number of scan lines is from 11 to 15, then thedepth layer level can be specified as 2 that is deeper than the depthlayer level 1.

Thus, with this image display device 10, the single photodetector 4 canbe used both to detect the touch coordinate and to detect themulti-stage depth. Thus, the number of parts can be reduced, whichlowers the cost.

Here, if the scanning of the laser light in the vertical direction bythe infrared laser unit 3 is performed at a constant speed, as shown inFIG. 6, then the scan lines become denser the lower down the scanning ison the touch surface T (i.e., the lower down in the vertical scanningdirection), at which the distance between the infrared laser unit 3 andthe touch surface T becomes shorter. Accordingly, if the depth layerlevels are specified with respect to each constant number of the scanlines, then the depth layer levels cannot be set at a position parallelto the touch surface T, as indicated by the one-dot chain line in FIG.6. Specifically, although the input object O advances to the same depth,the depth detection result by will vary with the position of the opticalimage S in the vertical direction. This can give an unnatural sensationto the user.

In view of this, alternatively or additionally, the depth detection canbe performed by the processing shown in FIG. 8. In step S1 in FIG. 8, ifthe controller 5 detects the touch coordinate and the number of the scanlines (Yes in step S1), then the flow proceeds to step S2. In step S2,the controller 5 acquires the depth layer level from a depth conversiontable stored ahead of time in the controller 5, based on the countednumber of scan lines and the vertical coordinate out of the detectedtouch coordinate.

FIG. 9 illustrates an example of the depth conversion table. As shown inFIG. 9, the correlation between the number of scan lines and the depthlayer level is specified for each vertical coordinate. In FIG. 9, thegreater is the value of the vertical coordinate, the lower is theposition on the touch surface T in FIG. 6. Also, the greater is thevalue of the vertical coordinate, the greater is the number of scanlines at a given depth layer level. Consequently, as shown by theone-dot chain line in FIG. 7, the depth can be set parallel to the touchsurface T. Therefore, the depth detection result can be the sameregardless of the vertical direction position when the input object Omoves while maintaining a given depth. Thus, this can suppress theunnatural sensation of the user.

After step S2, in step S3, the controller 5 outputs the detected touchcoordinate and the acquired depth layer level. Then, the processing inFIG. 8 is ended.

Only the vertical coordinate is used above. However, alternatively, onlythe horizontal coordinate can be used, or both the vertical coordinateand the horizontal coordinate can be used.

Next, an application example of using the touch coordinate detection andthe depth detection will be described through reference to FIGS. 10 to12. The application described here is a drag mode in which the inputobject O is pushed in.

At the start of the flowchart shown in FIG. 11, in step S11, the inputobject O, such as the finger and the like, touches the touch surface T(e.g., the state (A) in FIG. 10). When the controller 5 detects thetouch coordinate (Yes in step S11), the flow proceeds to step S12.

In step S12, the controller 5 detects the depth layer level. If thedetected layer level is determined to be at least a specific layerlevel, and the push-in of the input object O has been detected (Yes instep S12), then the flow proceeds to step S13 (e.g., the state (B) inFIG. 10). In step S13 the controller 5 changes to drag mode.

On the other hand, in step S12, if the controller 5 determines that thedetected layer level has not reached the specific layer level, and theinput object O has not been pushed in (No in step S12), then the flowproceeds to step S14. In step S14 the controller 5 performs processingas if a tap input has been performed. For example, if the finger touchesthe icon I and the finger is stopped in that state, as in the state (A)in FIG. 10, then the operation is performed as the tap input.

When the mode changes to the drag mode, the flowchart in FIG. 12commences. In step S15, if the controller 5 detects the touch coordinate(Yes in step S15), then the flow proceeds to step S16. In step S16, ifit is determined that the depth layer level detected along with thetouch coordinate in step S15 is at least a specific layer level (Yes instep S16), then the flow proceeds to step S17. In step S17, thecontroller 5 controls the display of the liquid crystal displaycomponent 2 so as to move the icon to the position of the touchcoordinate detected in step S15. The moved icon is an icon that includesthe touch coordinate detected in step S11 (see FIG. 11). After step S17,the flow returns to step S15.

If the finger is pushed into the icon I to change to the drag mode, asin the state (B) in FIG. 10, then steps S15 to S17 are repeated. As aresult, when the finger is moved while still pushed in, the icon Ifollows this movement, which results in the state shown in the state (C)in FIG. 10.

Then, in step S15, if the controller 5 cannot detect the touchcoordinate (No in step S15), or if the touch coordinate is detected instep S15, but the specific layer level is not reached in step S16 (No instep S16), then the flow proceeds to step S18. In step S18, thecontroller 5 controls the display of the liquid crystal displaycomponent 2 so as to fix the icon.

Consequently, if the finger is pull away from the icon I as in the state(D) in FIG. 10 after the icon I has been dragged as in the state (C) inFIG. 10, then the icon I is fixed (i.e., the icon is dropped) by stepS18.

Thus, a user interface in which the icon can be dragged over the opticalimage S can be achieved by the touch coordinate detection and the depthdetection in this embodiment. Therefore, the icon can be manipulatedmore intuitively by the user.

Next, a pull-out operation with the input object O will be described asan application example. In this embodiment, as shown in FIG. 13, theimage display device 10 further includes a photodetector 15 (e.g., asecond photodetector) separately from the photodetector 4. Thephotodetector 15 has a detection range R2 (e.g., a second detectionrange) as illustrated with one-dot chain lines in FIG. 13. The detectionrange R2 of the photodetector 15 is set more to the front side, as seenfrom the direction in which the input object O advances into the opticalimage S, than the detection range R1 of the photodetector 4 asillustrated with broken lines in FIG. 13. The photodetector 15 issimilar to the photodetector 4 in that it includes a light receivingelement 15A, a converging lens 15B, and a masking member (not shown)disposed therebetween. The detection signal from the light receivingelement 15A is sent to the controller 5 (see FIG. 2).

The pull-out operation processing with this configuration will bedescribed through reference to FIGS. 14 and 15. At the start of theflowchart in FIG. 15, in step S21, if the controller 5 detects two touchcoordinates (Yes in step S21), then the flow proceeds to step S22. Forexample, as shown in the state (A) in FIG. 14, the two touch coordinatesare detected if two fingers touch the optical image S.

In step S22, if the controller 5 determines that the distance betweenthe two detected touch coordinates is within a specific distance (Yes instep S22), then the flow proceeds to step S23. For example, if the twofingers move closer as in the state (B) after the state (A) in FIG. 14,then the distance between the two touch coordinates is within thespecific distance.

Then, in step S23, the controller 5 continues detecting the touchcoordinates. If it is determined that the photodetector 4 no longerdetects the reflected laser light from the input object O (Yes in stepS23), then the flow proceeds to step S24.

In step S24, the controller 5 detects the position coordinates on thetouch surface T for each laser light based on the values of the verticaland horizontal synchronization signals according to the timing at whichthe laser lights reflected by the tips of two input objects O aredetected by the photodetector 15. If the distance between the detectedposition coordinates is within a specific distance (Yes in step S24),then the flow proceeds to step S25. In step S25, the controller 5determines that the pull-out operation is performed.

For example, after the state (B) in FIG. 14, if the two fingers aremoved closer or pulled forward as in the state (C), then after theprocessing in steps S23 and S24, it is determined in step S25 that thepull-out operation is performed.

In step S25, the controller 5 detects the position coordinate of thelaser light on the touch surface T based on the values of the verticaland horizontal synchronization signals according to the timing at whichthe laser light reflected by the tip of the input object O is detectedby the photodetector 15. The pull-out amount of the input object O iscalculated based on this detected position coordinate and the touchcoordinate detected last when the detection of the touch coordinates iscontinued in step S23.

The controller 5 controls the display of the liquid crystal displaycomponent 2 so as to expand or shrink the image at the optical image Saccording to the pull-out amount, for example.

Thus, in this embodiment, a user interface in which input by thepull-out operation with the two input objects O can be achieved, whichmakes input more intuitive for the user.

An embodiment of the present invention is described above. However,various modifications to the embodiment are possible without departingfrom the scope of the present invention.

For example, the light beam scanned two-dimensionally is not limited tothe infrared light, and can be a visible light. With this configuration,since the user can see the color when the visible light is reflected bythe input object O, such as a finger, the user can recognize that theinput object O is located in the scanning range of the light beam forsure.

In the illustrated embodiment, the image display device 10 (e.g., thespatial input device) includes the infrared laser unit 3 (e.g., thelight beam scanner), the photodetector 4 (e.g., the photodetector), andthe controller 5 (e.g., the controller). The infrared laser unit 3 isconfigured to emit laser beams (e.g., light beams) toward the spatiallyprojected optical image S (e.g., the image) while two-dimensionallyscanning the laser beams. The photodetector 4 is configured to detectthe laser beams that have been reflected by the input object O, such asthe finger, within the detection range R1. The detection range R1extends inward of the image display device 10 relative to the opticalimage S. The controller 5 is configured to count the number of scanlines (e.g., the scan line number) indicative of the number of the laserbeams that have been detected by the photodetector 4, and to detect thedepth position of the input object O based on the number of scan line.In the illustrated embodiment, the controller 5 can perform the countingof the number of scan lines and the detecting of the depth position witha depth detector or detecting component that can be realized anycombination of hardware and software.

With this configuration, the depth position of the input object O thathas moved in deeper than the optical image S can be detected accuratelyand quickly by using the single photodetector 4. Therefore, the depthposition can be detected accurately and quickly with a simple structure.

In the illustrated embodiment, the photodetector 4 is arranged such thatone end of the detection range R1 is arranged to coincide with or beparallel to the optical image S, and such that the other end of thedetection range R1 is disposed on a far side of the image display device10 relative to the optical image S along the advance direction (e.g.,the approach direction) of the input object O relative to the opticalimage S.

In the illustrated embodiment, the controller 5 is further configured todetect as the touch coordinate of the input object O the irradiationposition coordinate of one of the laser lights that is finally orinitially detected by the photodetector 4 at one end of the detectionrange R1 during the scanning of the infrared laser unit 3 based on thetiming at which the photodetector 4 detects the one of the laser lights.In the illustrated embodiment, the controller 5 can perform thedetection of the touch coordinate with a touch coordinate detector ordetecting component that can be realized any combination of hardware andsoftware.

With this configuration, in addition to the depth position, the positionwhere the optical image S has been touched by the input object O canalso be detected by using the single photodetector 4. This lowers themanufacturing cost.

In the illustrated embodiment, the controller 5 is further configured todetect the depth position based on the number of scan lines such that acorrelation between the number of scan lines and the depth positionvaries according to the touch coordinate.

With this configuration, even if the density of the scan lines variesaccording to the scanning position, deviation or variance in the depthdetection result depending on the touch position can be suppressed. Thismakes the operation feel more natural to the user.

In the illustrated embodiment, the controller 5 is further configured toswitch between a plurality of processings based on whether or not thedepth position (e.g., the depth layer level) is larger than a specificdepth position (e.g., the specific layer level) in response to the touchcoordinate being detected. In the illustrated embodiment, the controller5 can perform the switching of the processings with a processingcomponent that can be realized any combination of hardware and software.

With this configuration, the processings can be changed between a casein which the input object O has advanced deeper than the optical imageS, and the input object O has been pushed in further, and a case inwhich the input object O has been stopped at the specific depthposition.

In the illustrated embodiment, the controller 5 is further configured tomove the icon I (e.g., the part of the image) to the touch coordinate inresponse to determining that the depth position is larger than thespecific depth position.

With this configuration, if the input object O has advanced deeper thanthe optical image S, and the input object O has been pushed in further,then the mode can be changed to the drag mode in which the icon I orother specific image is made to follow the position of the input objectO.

In the illustrated embodiment, the controller 5 is further configured todetermine that the pull-out operation is performed in response to thephotodetector 4 no longer detecting the touch coordinate after aplurality of touch coordinates have been detected. In the illustratedembodiment, the controller 5 can perform the determination of thepull-out operation with a determination component that can be realizedany combination of hardware and software.

With this configuration, an operation in which a plurality of inputobjects O have advanced deeper than the optical image S and are thenpulled out toward the user can be detected as the pull-out operation.

In the illustrated embodiment, the image display device 10 furtherincludes the photodetector (e.g., the second photodetector) configuredto detect the laser lights that have been reflected by the input objectO within the detection range R2 (e.g., the second detection range). Thedetection range R2 extends outward of the image display device 10relative to the detection range R2 of the photodetector 15.

Furthermore, in the illustrated embodiment, the controller 5 is furtherconfigured to determine that the pull-out operation is performed inresponse to the photodetector 4 no longer detecting the touch coordinateafter a plurality of touch coordinates have been detected. Moreover, thecontroller 5 is further configured to calculate the pull-out amount ofthe input object O based on a detection result of the photodetector 15in response to the controller 5 determining that the pull-out operationis performed. In the illustrated embodiment, the controller 5 canperform the calculation of the pull-out amount with a calculator orcalculation component that can be realized any combination of hardwareand software.

With this configuration, if the pull-out operation has been performed,then the pull-out amount can be calculated, and the operationcorresponding to the pull-out amount can be performed.

With the present invention, the depth position can be detectedaccurately and quickly with a simple structure.

In accordance with a first aspect, a spatial input device comprises: alight beam scanner configured to emit light beams toward a spatiallyprojected image while two-dimensionally scanning the light beams; aphotodetector configured to detect the light beams that have beenreflected by an input object within a detection range, with thedetection range extending inward of the spatial input device relative tothe image; and a controller configured to count a scan line numberindicative of a number of the light beams that have been detected by thephotodetector, and to detect a depth position of the input object basedon the scan line number.

In accordance with a second aspect, with the spatial input deviceaccording to the first aspect, the photodetector is arranged such thatone end of the detection range is arranged to coincide with or beparallel to the image, and such that the other end of the detectionrange is disposed on a far side of the spatial input device relative tothe image along an approach direction of the input object relative tothe image.

In accordance with a third aspect, with the spatial input deviceaccording to the first aspect, the controller is further configured todetect as a touch coordinate of the input object a irradiation positioncoordinate of one of the light beams that is finally or initiallydetected by the photodetector at one end of the detection range during ascanning of the light beam scanner based on a timing at which thephotodetector detects the one of the light beams.

In accordance with a fourth aspect, the spatial input device accordingto the third aspect, the controller is further configured to detect thedepth position based on the scan line number such that a correlationbetween the scan line number and the depth position varies according tothe touch coordinate.

In accordance with a fifth aspect, the spatial input device according tothe third aspect, wherein the controller is further configured to switchbetween a plurality of processings based on whether or not the depthposition is larger than a specific depth position in response to thetouch coordinate being detected.

In accordance with a sixth aspect, the spatial input device according tothe fifth aspect, the controller is further configured to move a part ofthe image to the touch coordinate in response to determining that thedepth position is larger than the specific depth position.

In accordance with a seventh aspect, the spatial input device accordingto the third aspect, the controller is further configured to determinethat a pull-out operation is performed in response to the photodetectorno longer detecting the touch coordinate after a plurality of touchcoordinates have been detected.

In accordance with an eighth aspect, the spatial input device accordingto the first aspect, further comprises a second photodetector configuredto detect the light beams that have been reflected by the input objectwithin a second detection range, with the second detection rangeextending outward of the spatial input device relative to the detectionrange of the photodetector.

In accordance with a ninth aspect, the spatial input device according tothe eighth aspect, the controller is further configured to determinethat a pull-out operation is performed in response to the photodetectorno longer detecting the touch coordinate after a plurality of touchcoordinates have been detected, and the controller being furtherconfigured to calculate a pull-out amount of the input object based on adetection result of the second photodetector in response to thecontroller determining that the pull-out operation is performed.

In understanding the scope of the present invention, the term“comprising” and its derivatives, as used herein, are intended to beopen ended terms that specify the presence of the stated features,elements, components, groups, integers, and/or steps, but do not excludethe presence of other unstated features, elements, components, groups,integers and/or steps. The foregoing also applies to words havingsimilar meanings such as the terms, “including”, “having” and theirderivatives. Also, the terms “part,” “section,” “portion,” “member” or“element” when used in the singular can have the dual meaning of asingle part or a plurality of parts unless otherwise stated.

As used herein, the following directional terms “forward”, “rearward”,“front”, “rear”, “up”, “down”, “above”, “below”, “upward”, “downward”,“top”, “bottom”, “side”, “vertical”, “horizontal”, “perpendicular” and“transverse” as well as any other similar directional terms refer tothose directions of an image display device in an upright position.Accordingly, these directional terms, as utilized to describe the imagedisplay device should be interpreted relative to an image display devicein an upright position on a horizontal surface.

Also it will be understood that although the terms “first” and “second”may be used herein to describe various components these componentsshould not be limited by these terms. These terms are only used todistinguish one component from another. Thus, for example, a firstcomponent discussed above could be termed a second component andvice-a-versa without departing from the teachings of the presentinvention. The term “attached” or “attaching”, as used herein,encompasses configurations in which an element is directly secured toanother element by affixing the element directly to the other element;configurations in which the element is indirectly secured to the otherelement by affixing the element to the intermediate member(s) which inturn are affixed to the other element; and configurations in which oneelement is integral with another element, i.e. one element isessentially part of the other element. This definition also applies towords of similar meaning, for example, “joined”, “connected”, “coupled”,“mounted”, “bonded”, “fixed” and their derivatives. Finally, terms ofdegree such as “substantially”, “about” and “approximately” as usedherein mean an amount of deviation of the modified term such that theend result is not significantly changed.

While only a selected embodiment has been chosen to illustrate thepresent invention, it will be apparent to those skilled in the art fromthis disclosure that various changes and modifications can be madeherein without departing from the scope of the invention as defined inthe appended claims. For example, unless specifically stated otherwise,the size, shape, location or orientation of the various components canbe changed as needed and/or desired so long as the changes do notsubstantially affect their intended function. Unless specifically statedotherwise, components that are shown directly connected or contactingeach other can have intermediate structures disposed between them solong as the changes do not substantially affect their intended function.The functions of one element can be performed by two, and vice versaunless specifically stated otherwise. It is not necessary for alladvantages to be present in a particular embodiment at the same time.Every feature which is unique from the prior art, alone or incombination with other features, also should be considered a separatedescription of further inventions by the applicant, including thestructural and/or functional concepts embodied by such feature(s). Thus,the foregoing descriptions of the embodiment according to the presentinvention are provided for illustration only, and not for the purpose oflimiting the invention as defined by the appended claims and theirequivalents.

What is claimed is:
 1. A spatial input device comprising: a light beamscanner that emits light beams toward a spatially projected image whiletwo-dimensionally scanning the light beams; a photodetector that detectsthe light beams that have been reflected by an input object within adetection range, with the detection range extending inward of thespatial input device relative to the image; and a controller that countsa scan line number indicative of a number of the light beams that havebeen detected by the photodetector, and detects a depth position of theinput object based on the scan line number, the controller furtherdetecting the depth position of the input object based on a depthconversion table, with the depth conversion table specifying depth layerlevels corresponding to scan line numbers.
 2. The spatial input deviceaccording to claim 1, wherein one end of the detection range is arrangedto coincide with or be parallel to the image, and the other end of thedetection range is disposed on a far side of the spatial input devicerelative to the image along an approach direction of the input objectrelative to the image.
 3. The spatial input device according to claim 1,wherein the controller further detects as a touch coordinate of theinput object an irradiation position coordinate of one of the lightbeams that is finally or initially detected by the photodetector at oneend of the detection range during a scanning of the light beam scannerbased on a timing at which the photodetector detects the one of thelight beams.
 4. The spatial input device according to claim 3, whereinthe touch coordinate is a two-dimensional coordinate having a verticalcoordinate and a horizontal coordinate.
 5. The spatial input deviceaccording to claim 3, wherein the controller further switches between aplurality of processings based on whether or not the depth position islarger than a specific depth position in response to the touchcoordinate being detected.
 6. The spatial input device according toclaim 5, wherein the controller further moves a part of the image to thetouch coordinate in response to determining that the depth position islarger than the specific depth position.
 7. The spatial input deviceaccording to claim 3, wherein the controller further determines that apull-out operation is performed in response to the photodetector nolonger detecting the touch coordinate after a plurality of touchcoordinates have been detected.
 8. The spatial input device according toclaim 3, wherein the depth conversion table specifies correlationbetween the scan line numbers and the depth layer levels according tothe touch coordinate.
 9. The spatial input device according to claim 1,wherein the controller further acquires a depth layer level from thedepth conversion table based on the counted scan line number and avertical coordinate out of the touch coordinate.
 10. The spatial inputdevice according to claim 9, wherein the depth conversion table dividesthe light beams into a plurality of beam groups each of which has thesame scan line number, and specifies the depth layer levels with respectto the plurality of beam groups.
 11. The spatial input device accordingto claim 9, wherein the depth conversion table divides the light beamsinto a plurality of beam groups the scan line number of which changes inaccordance with increase or decrease of the vertical coordinate and/orthe horizontal coordinate specified thereto, and specifies the depthlayer levels with respect to the plurality of beam groups.
 12. Thespatial input device according to claim 1, wherein the depth conversiontable specifies correlation between the scan line numbers and the depthlayer levels for each vertical coordinate.
 13. The spatial input deviceaccording to claim 1, wherein the controller further acquires a depthlayer level from the depth conversion table based on the counted scanline number and a horizontal coordinate out of the touch coordinate. 14.The spatial input device according to claim 1, wherein the depthconversion table specifies correlation between the scan line numbers andthe depth layer levels for each horizontal coordinate.
 15. The spatialinput device according to claim 1, further comprising a secondphotodetector that detects the light beams that have been reflected bythe input object within a second detection range, with the seconddetection range extending outward of the spatial input device relativeto the detection range of the photodetector.
 16. The spatial inputdevice according to claim 15, wherein the controller further determinesthat a pull-out operation is performed in response to the photodetectorno longer detecting the touch coordinate after a plurality of touchcoordinates have been detected, and the controller further calculates apull-out amount of the input object based on a detection result of thesecond photodetector in response to the controller determining that thepull-out operation is performed.
 17. The spatial input device accordingto claim 1, wherein the depth position of the input object is indicativeof an insertion amount of the input device that has been inserted withinthe detection range beyond an image plane defined by the image.
 18. Thespatial input device according to claim 1, wherein the depth position ofthe input object is indicative of a far side position of the inputobject relative to the image in a direction perpendicular to the image.19. The spatial input device according to claim 1, wherein the depthconversion table specifies the depth layer levels non-parallel to animage plane defined by the image.
 20. The spatial input device accordingto claim 1, wherein the depth conversion table specifies the depth layerlevels parallel to an image plane defined by the image.